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ABSTRACT 

The purpose of this paper is to discuss the reasons 
for resistance to evaluation and the factors that affect adoption of 
new procedures, while focusing on a model for guiding the rlanning of 
evaluation implementation strategies. The process of adoption of an 
innovation occurs in three stages: orientation, where the user 
identifies benefits; initiation, where the user gives an innovation a 
trial; and integration, where it becomes i>art of routine practice. At 
each stage in the adoption, different strategies can be directed to 
gain the cooperation of either the user of the evaluation or the 
organization: (1) power; (2) communication; (3) consultation; (4) 
reinforcement; (5) training ; and (6) direct measures. The strategies 
differ depending upon the time frame in which they are applied. That 
is, different operations are required prior to commencement, during 
the initiation of the evaluation, and when evaluation procedures are 
to be integrated into routine organizational activities. It is 
concluded that: (1) in planning the implementation of evaluation, the 
implications are that implementation strategies should persuade 
rather than enforce cooperation; (2) cognitive and perceptual changes 
must be accompanied by behavioral changes; and (3) different 
management practices will be applicable at different time intervals 
in the evaluation procedure. (PN) 
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ABSTRACT 

This paper presents a model for guiding the procedures involved in implementing 
evaluation* It describes six strategies directed towards gaining access to and the 
co-operation of individuals and the organization in which evaluation is to be 
introduced. 

The strategies differ depending upon the time frame in which they are applied. That 
is, different operations are required prior to the commencement, during the initiation 
of the evaluation, and when evaluation procedures are to be integrated into routine 
organizational activities. 



INTRODUCTION 



Criticisms of evaluation revolve around political, social, psychological and 
implementation problems. In the 1980s practitioners met the criticisms by new 
paradigms which tcok account of the social context in which programs operate, the 
complexity o! the problems being addressed and tne constraints posed by the human 
element in organizational processes and structures. Models of evaluation emerged 
that attempted to explain the context of evaluation (Stuff lebeam 1983), the 
relationship between system elements (Borich and Jemelka 1982, Armstrong 1983, 
Wholey 1977), the role of evaluation providing different feedback loops to different 
decision hierarchies in the program system (Armstrong 1985), the conflicting values, 
expectations and purposes of the audiences that evaluation is expected to serve and 
the use of evaluation results (Patton 1978, Si ±e 1975, MacDonald 1985, Scriven 1980, 
Weiss 1985, Jamrozik 1985), 

The purpose of this paper is to discuss the reasons for resistance to evaluation and 
the factors that affect adoption of new procedures, and piesent a model for guiding 
the planning of evaluation implementation strategies. 




RESISTANCE TO EVALUATION 

Evaluation is disruptive, innovative and a vehicle of change. People resist change 
which ihreatens their basic security, that they do not understand, and when it is 
forced upon them. (Spicer 1952). Resistance occurs when those affected by change 
perceive it as threatening, when it challenges currently held beliefs and values or 
when it changes the distribution of power and influence. 

Evaluation is threatening when it arouses fear. Research shows that the fears are 
associated with threats to job security, loss of prestige or status, devaluation of 
current knowledge or skills, or the highlighting of lack of appropriate skills 
(NIMH 1973). 

Fear of loos of self-esteem or sense of competency and/or fear of exposure of weak 
points has been associated with learning new methods of won: and challenges to 
already learnt theoretical frameworks and practices. Alternatively, difficulties have 
arisen because of evaluators 1 lack of recognition of current successful efforts. 

The purpose of evaluation implementation strategies is to persuade people that the 
rewards of evaluation outweigh the reasons for resistance. They are intended to 
dispel fear, develop understanding of the evaluation, encourage ownership of and 
commitment to the evaluation, and ensure that the evaluation results will be used. 

FACTORS AFFECTING THE ADOPTION OF EVALUATION 

Weiss (1985) identifies several individual and organizational factors that inhibit the 
use of evaluation including intellectual-cognitive limits due to disciplinary and 
language barriers between the program staff and evaluators; the difficulty of knowing 
which evaluation information will serve decisions or what are the critical issues; the 
lack of appropriate dissemination of evaluation results; and the fragmentation of 




authority, i.e. decision making is usually incremental and there is often no 
commitment and little authority to implement the results. 

Successful organizations are facing challenges to come up with new ideas, to develop 
creative responses to change, and to show more innovation, enterprise and initiative 
(Kanter 1985). The introduction of evaluation procedures and the implementation of 
action plans arising from evaluation is an innovative response to the pressures 
experienced by organizations in today f s fast changing, competitive and uncertain 
environment. 

The failure of innovations is due to lack of clarity about the innovation, lack of 
capability to perform a required task, lack of necessary support materials, 
incompatibility with organizational arrangements, inadequate resources, distractions 
from training in implementing the innovation, and lack of implementation training 
(Roberts-Gray and Gray 1983). 

The adoption of innovation, that is, the practical application of ideas and the use of 
research results, were explored by Rogers and Shoemaker (1970). 

The factors affecting adoption of an innovation are: 

The oower leaders. 

Recognized leaders with power and influence can influence adoption. 
Without such support to give the organization time to adjust, innovators 
usually succumb to the hostility of opponents of change (Lippitt 1965). 

The source of the message. 

The more prestigious the source of the message, the more notice is taken 
of the message. Higher credibility is attached to a source seen as an 
expert in the field and as having no vested interest in the outcome of the 
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3. Participation in decision-making. 

Studies of participatory and industrial democracy have shown that workers ar* 
committed to situations in which they have a major influence on the decisions. 

4. The characteristics of the adopting people. 

Rogers (1962) identified five classes of adcptors bas?d upon their degree of 
innovativeness: innovators (venturesome); early adoptors (respectful); early 
majority (deliberate); later majority (sceptical); and laggards (traditional). Early 
adoptors tend to be more modern, better educated, have greater rationality and 
a more favourable attitude towards change and risk. 

5. Communication channels. 

The preferred communication channels for innovations are those that are close 
in contact with the origin of the new ideas. The mass media is appropriate for 
the early adoptors and the majority. Inter-personal communication from peers 
is important for laggards. 

6. Organization context and social norms. 

Innovation is more likely to be accepted if it is seen as meeting some need in 
the organization and as fitting in with the current values and experienced 
members of the organization. Consequently there is likely to be a greater 
acceptance in times of crisis (Havelock 1969). 

7. Capability. 

The successful implementation of an innovation depends upon the capability of 
both the users and the organization. User capability refers to the knowledge, 
skills and expertise of individual members of the organization. Organization 
capability refers to the facilities, structure, rules, etc. that enable 
implementation to occur. (Roberts-Gray and Gray 1983). 
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8. Organization structure. 

Kanter (1985) identified the differences between the structure of companies that 
were closed or open to innovation. Segmented organizations are likely to be 
anti-change orientated and prevent innovation. Segmentalisjn "is concerned with 
compartmentalising actions, events and problems and keeping each piece isolated 
from the others". Segmentanst structures occur when departments are "walled 
off from one another, level above from level below, field officer from 
headquarters, labour from management or men from women" (p.28). In such 
companies problems are narrowly defined independently of their context and 
connections to other problems. It is assumed that problems can be solved when 
they are carved into pieces and the pieces assigned to specialists to work on in 
isolation. 

As any change threatens to disturb the regularity of segments the system is 
designed to protect itself against deviations and ensure that individuals have 
sufficient awe and respect to maintain their roles without question. 

Segmentalism inhibits innovation because segmentalism discourages people from 
seeing and hence being motivated to solve problems. It discourages 
entrepreneurial activities because traditional rather than novel ways of coping 
are preferred. Furthermore, specialized biases and political conflicts are likely 
to inhibit innovation and co-operation. 

In contrast, integrative structures encourage the "treatment of problems as 
wholes" considering the wider implications of actions. Such organizations reduce 
rancorous conflict and isolation between organizational units, create mechanisms 
for the exchange of new ideas and information across organizational boundaries, 
ensure that multiple perspectives will be taken into account in decision making 
and provide coherance and direction to the whole organization. In team 
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orientated, co-operative environments, innovation flourishes." (p.28). 

The organization open to innovation is less category-conscious, has a large 
number of integrative mechanisms encouragirg fluidity of boundaries, the free 
flow of ideas and empowerment of people to act on new information. There is 
likely to be a sense of unity and identification with the organization, respect 
for individuals within the organization, and reward systems that tend towards 
investment-in-the-future orientation. 

MOTIVATION TO UTILIZE EVALUATION RESULTS 

Individuals use evaluation. Their behaviour is determined by the relative importance 
of attitudinal and normative considerations (Fishbein and Ajzen 1975). 

Attitudes are shaped by a person f s beliefs that the behaviour leads to certain 
outcomes and his evaluation of these outcomes. Attitude towards evaluation will be 
determined by (a) the information available, e.g. the benefits of the evaluation, and 
(b) an assessment of the value of the information, e.g. will the evaluation lead to the 
benefits and will the benefits advantage the individual. 

The subjective norm is the result of the person f s beliefs that specific individuals or 
groups think that he should or should not perform the behaviour (e.g. peer views about 
the evaluation) and his motivation to comply with the specific referents (e.g. his 
desire for inclusion in the work group, awareness that promotion will result from 
implementation of evaluation results). 
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IM PLICATIONS FOR PLANNING EVALUATIONS 

The implications of the above research into resistance to change, factors that affect 
the adoption of innovations, and the prediction of behaviour is that evaluation is more 
likely to be successful if it is adopted first by innovators and opinion leaders, it is 
introduced by independent experts in evaluation, people impacted by the evaluation 
have some say and involvement in its introduction, if the organizational climate 
is such that leaders endorse the introduction of evaluation, the evaluation is seen to 
meet some need, if the organization is open to new ideas and flows of information, 
and if the evaluation is integrated into overall organization functions. Supporc for 
the evaluation may take the form of facilities, resource allocation, new management 
structures, etc. 

Participation in the evaluation would be encouraged by getting the people who are 
to be evaluated or who deliver the services involved in the process of evaluation. 
They could define problems, identify needs, carry out evaluation tasks and disseminate 
and utilize the results. 

Individuals are more likely to participate in and use evaluation if they perceive 
benefits to themselves and are motwated to comply by leaders and peer pressures. 

A condition of their involvement and commitment may be the acquisition of skills 
required in evaluation (such as those of negotiation, facilitation, data collection, etc.). 
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AN IMPLEMENTATION MODEL 

The process ol adoption of an innovation occurs in stages Lewin (1962) identifies 
three stages: orientation, where the user identifies benefits; initiation, where the 
user gives an innovation a trial, evaluates the benefits and sees how it can be 
integrated into routine practice; and integration, where it becomes part of routine 
practice. At each stage in the adoption of evaluation procedures, different strategies 
can be fooussed either on the user of the evaluation or the organization. (Figure 1), 



STRATEGIES FOR IMPLEMENTI NG EVALUATION 

There are at least six strategies for successfully implementing program budgeting and 
evaluation: power, communication, consultation, reinforcement, training and direct 
measures. Each strategy is applied in different ways at each stage in the 
implementation process, (Table 1), 



Power/coercive strategies emphasize political and economic sanctions. Leadership 
endorsement is essential in the orientation stage. Leaders must demonstrate that 
they want evaluation and maintain a continuing series of reinforcing messages. 
Kanter (1385) refers to the symbolic aspects of strategy after events are in motion. 
The views and expectations of stakeholders should be canvassed, and later, formal 
agreement is reached. Funding or promotion may depend on co-operation or more 
funding may be channelled to departments which volunteer to introduce the program. 
In the individual user, peer or union pressures may affect adoption. Finally, the 
formal rules of the organization will integrate the operation of the new evaluation 
procedures. 
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Communication of information is important to both the evaluator and the program 
staff. The evaluator needs to know the objectives of the evaluation, the terms of 
reference (if there are any) and to understand the context of the program. The user 
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STAGES IN AOuPTION PROCESS 



FIGUKE 1. IMPLEMENTATION STRATEGIES 
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TABLE 1. STRATEGIES AND ACTION PLANS FOP MANAGING THE HUMAN ELEMENT IN EVALUATION 
STRATEGIES ORIENTATION INITIATION INTEGRATION 



Power 



Leadership 
Endorsement 



Letters of agreement 
Basis of funding 



m 



Oi ganisational 
rules 



Communication 



Information for e valuator 

(context) 
Information for user 

(Benefits) 



Opinion leaders bulletins 

Wide dissemination meet 
crititisms/real grievances 



Routine 

Information systems 



Consultation 



Person to Person/ group 

meetings 

Involve users 



Group meetings 
Evaluator support 



Feedback 
Interpretation 



Reinforcement 



Marketing News 



Trial 

Peer pressures 
On going feedback 
Staff development experiences 



Recognition 
Promo t ion /rewards on 
basis of evaluation 



Training 



Evaluation SKills 



Dissemination/ 
Interpretation skills 



Using evaluation/ 
Information skills 



Direct 
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Direct resojrees to 
evaluation 

Links with evaluator 
and users 



Link evaluation and 
and planning 

Develop data information 
system 



Evaluation regular 
ac t ivi ty 

Responsibility to 
user 
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needs realistic information about: the benefits of using evaluation results (what's in 
;l for me?), security arrangements, the purpose of the evaluation and now to use the 
results in decision making. At the initiation stage the evaluator will make direct 
contact with opinion leaders (the venturesome) and mass media (e.g. the organisation's 
newsletter) will alert the majority of users (the deliberate and sceptical innovators). 
The aiir at this stage is to develop an understanding of and expectation for change. 
At the integration stage the use of evaluation will be routinized. 

In addition to information, the user may need person to person or group consultation 
to encourage commitment, understanding and to reduce the threat of evaluation. 
Feedback on their efforts is essential. 

Reinforcement means providing rewarding experiences. Initially it may mean 
identifying with the new innovation described in the organization's marketing news. 
At the initiation stage, staff development experiences and recognition will enhance 
adoption. Particularly, in formative evaluation, focussing on identifying critical success 
factors in the program and avoiding a singular crusade to find the trouble spots will 
be more succes c ful in improving the program without reducing staff morale to zero. 
Integration occurs when th^ use of evaluation is seen as a basis for promotion. 

Although many government departments in Australia have introduced evaluation 
programs, few devote the necessary resources to ensuring that the user receives the 
training required for successful implementation and use of results. 

Finally, the direct action strategy requires that the organization devotes adequate 
resources to the implementation process, to developing links between policy analysts, 
data analysts, the program staff or other users. This model of implementation is 
based on a philosophy of fostering autonomy and responsibility in management. An 
organization with centralized authoritative and segmented structures may consider 
modification of existing management practices. The integration of these activities 
mean that the evaluation program becomes a regular monitoring activity. A 5 



CONCLUSION 

In planning the implementation of evaluation the implications are (1) implementation 
strategies should persuade rather than enforce co-operation, (2) cognitive and 
perceptual changes must be accompanied by behavioural changes, and (3) different 
management practices will be applicable at different time intervals in the evaluation 
procedure. 
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